Dual Space Preconditioning for Gradient Descent
نویسندگان
چکیده
The conditions of relative smoothness and strong convexity were recently introduced for the analysis Bregman gradient methods convex optimization. We introduce a generalized left-pr...
منابع مشابه
Dual Space Gradient Descent for Online Learning
One crucial goal in kernel online learning is to bound the model size. Common approaches employ budget maintenance procedures to restrict the model sizes using removal, projection, or merging strategies. Although projection and merging, in the literature, are known to be the most effective strategies, they demand extensive computation whilst removal strategy fails to retain information of the r...
متن کاملNonsymmetric Preconditioning for Conjugate Gradient and Steepest Descent Methods1
We numerically analyze the possibility of turning off postsmoothing (relaxation) in geometric multigrid when used as a preconditioner in conjugate gradient linear and eigenvalue solvers for the 3D Laplacian. The geometric Semicoarsening Multigrid (SMG) method is provided by the hypre parallel software package. We solve linear systems using two variants (standard and flexible) of the preconditio...
متن کاملSteepest Descent and Conjugate Gradient Methods with Variable Preconditioning
We analyze the conjugate gradient (CG) method with variable preconditioning for solving a linear system with a real symmetric positive definite (SPD) matrix of coefficients A. We assume that the preconditioner is SPD on each step, and that the condition number of the preconditioned system matrix is bounded above by a constant independent of the step number. We show that the CG method with varia...
متن کاملL1 Regression using Lewis Weights Preconditioning and Stochastic Gradient Descent
We consider the l1 minimization problem minx ‖Ax − b‖1 in the overconstrained case, commonly known as the Least Absolute Deviations problem, where there are far more constraints than variables. More specifically, we have A ∈ R for n ≫ d. Many important combinatorial problems, such as minimum cut and shortest path, can be formulated as l1 regression problems [CMMP13]. We follow the general parad...
متن کاملLearning by Gradient Descent in Function Space
Traditional connectionist networks have homogeneous nodes wherein each node executes the same function. Networks where each node executes a di erent function can be used to achieve e cient supervised learning. A modi ed back-propagation algorithm for such networks, which performs gradient descent in \function space," is presented and its advantages are discussed. The bene ts of the suggested pa...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Siam Journal on Optimization
سال: 2021
ISSN: ['1095-7189', '1052-6234']
DOI: https://doi.org/10.1137/19m130858x